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Abstract 

This study investigates the use of pragmatic markers (PM s) by learners of English 
at varying proficiency levels. The study analyzes data from a university-level oral 
proficiency exam that categorized Chinese and Korean English-as-a-second-lan- 
guage (ESL) speakers into four proficiency levels and compares data with those of 
native speakers taking the same test. Findings indicate that PM use generally rises 
with proficiency level. The rates of PM use showed a dramatic increase between 
the highest and second-highest proficiency group. The highest proficiency ESL 
group used PM s at the same rate as native speakers. The study also found that 
the variety of different PM s used goes up steadily with proficiency level. These 
results are discussed in terms of their implications for understanding how second 
language learners' use of PM s develops. 

Keywords: pragmatic marker, proficiency level, pragmatic competence, sec¬ 
ond language development, oral speech production 


1. Introduction 

Pragmatic makers (expressions such as so, well, and you know) have been shown 
to perform a number of important discourse and pragmatic functions in second 
language speech. In particular, as a subcategory of formulaic language in second 
language development, pragmatic markers facilitate the development of second 
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language fluency (Hasselgreen, 2004; Towell, Hawkins, & Bazergui 1996). They 
also allow for the communication of speakers' attitudes, how they intend their 
utterance to be interpreted, and the establishment of intersubjectivity with their 
interlocutor (Aijmer, 2013; Blakemore, 2008; Overstreet & Yule, 1997). As Halli- 
day and Hasan (1976) and Schiffrin (1987) point out in their analyses of LI prag¬ 
matics, pragmatic markers (PM s) are also one of the principal ways that coherent 
discourse is constructed (Halliday& Hasan, 1976; Schiffrin, 1987). 1 

Despite the wide variety of functions and their importance in the devel¬ 
opment of second language fluency, they have received relatively little atten¬ 
tion in second language acquisition research (M uller, 2005). 2 Previous research 
has examined the use of PMs in learner speech, but a number of important 
questions remain. In particular, little is known about the overall pattern of PM 
use across a wide spectrum of language learners. Although some L2 studies of 
PM s reveal important findings about individual markers (e.g., House, 2013; M ul¬ 
ler, 2005; Romero-Trillo, 2002), few studies focus on the wider range of use of 
PM s, including those that may be used less frequently by some learners. M ore- 
over, as Hasselgreen (2004) and Hellermann and Vergun (2007) note, the profi¬ 
ciency level of learners plays a role in the frequency and range of these types of 
expressions, although few studies break down proficiency levels into sub-levels 
while examining the larger set of PM s. Furthermore, few studies focus on the 
similarities and differences between highly advanced learners and native speak¬ 
ers with respect to the frequency and variety of PMs. Such information may tell 
us much about the interaction of proficiency level and pragmatic competence 
and whether learners progress more gradually or rapidly in the advanced stages 
of second language learning. 

This study attempts to shed new light on these issues, examining PM use 
among English as a second language (ESL) learners at multiple proficiency levels 
compared to native speakers performing the same tasks. The rate of use and 
the range, or richness, of PM use will be explored from a quantitative perspec¬ 
tive. The paper fills several gaps in the literature because it will examine the 
larger set of PM s rather than a small subset, and because it focuses on both the 
range and frequency of unique PM s across several sub-levels of proficiency. The 
results of statistical analysis reveal that the overall frequency of PM s increases 
at varying rates at each proficiency level and reaches a rate almost identical to 


1 Note that these authors use the term discourse markers rather than pragmatic markers, 
the term used throughout this paper. Further discussion of definitions used in this study is 
included in the background section on terminology. 

2 For a helpful overview of previous research on the development of L2 pragmatic compe¬ 
tence in general, see Bardovi-Harlig (2013) or Ishida (2013), in which the study of pragmatic 
markers is mentioned in the larger context of L2 pragmatics research. 
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that of native speakers, although the range remains relatively limited at the 
most advanced stage of proficiency. This study will explore possible reasons for 
this pattern of use and point to future studies that may reveal more about the 
development of L2 pragmatic competence. 

2. Review of the literature 

2.1. Pragmatic markers: Terminology and definitions 

Although PMs have received a great deal of attention in cross-linguistic studies 
over the last two decades, there is little consensus regarding the definition or ap¬ 
propriate terminology for this type of expression (Blakemore 2002, 2008; 
Romero-Trillo, 2012; Schourup 1999). 3 In an early study investigating discourse 
markers in English, Schiffrin (1987) uses this term to refer to "sequentially de¬ 
pendent elements that bracket units of talk" (p. 31). Fraser (1999) also uses the 
term discourse markers but defines them differently as "linguistically encoded 
clues which signal the speaker's potential communicative intention" (p, 168). Un¬ 
der the name discourse particles, they have been identified as particles that "are 
placed with great precision at different places in the discourse and give important 
clues to how discourse is segmented and processed" (Aijmer, 2002, p. 1). Alter¬ 
natively, others such as Hasselgreen (2004) define these expressions as small- 
words, which she points out are phrases "occurring with high frequency in the 
spoken language, that help to keep our speech flowing, yet do not contribute es¬ 
sentially to the message itself" (p. 162). These various terms and definitions are 
offered here to give some idea of the variety of approaches to these expressions. 

One of the most problematic aspects of classification deals with the dif¬ 
ference between what is referred to as pragmatic markers and what others 
might label discourse markers. This terminological confusion arises, according 
to Romero-Trillo (2012), because some scholars make clear distinctions be¬ 
tween the two terms while others use the term pragmatic markers to refer to 
a superordinate category under which discourse markers may be subsumed. In 
the present study, this more general term pragmatic marker will be used in or¬ 
der to avoid confusion with specific terms, including discourse markers, which 
refer to various subclasses of these expressions. In particular, the current study 
follows the definition and list used by Carter and M cCarthy (2006), in which the 


3 For further discussion of definitions and terminology of pragmatic markers, see Lenk 
(1997), Schoroup (1999), Callies (2009), Romero-Trillo (2012) and Aimjer (2013). Indepth 
studies of particular pragmatic markers have been undertaken in languages as diverse as 
Spanish (Chodorowska-Pilch, 2008; Duran & Unamuno, 2001), Chinese (Feng 2008; Wang& 
Tsai, 2007), Japanese (Sasamoto, 2008), and English (Bell, 2010; Lenk, 1998; Schourup, 1999). 
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term pragmatic marker is used. This definition and the reasons for its use will 
be discussed in the methodology section of this paper. 

Despite the lack of consensus concerning terminological conventions in 
previous research, according to Schourup (1999), there is general agreement on 
several fundamental characteristics of PMs. In his comprehensive overview, 
Schourup highlights three characteristics that most definitions of PMs share: 
connectivity, optionality, and non-truth-conditionality (pp. 1230-1232). Con¬ 
nectivity is the idea that PM s connect blocks of language to other blocks of lan¬ 
guage and that they signal the relationship between them. Optionality refers to 
the fact that these expressions are syntactically detachable from the utterance; 
they can be removed without disturbing the grammaticality of the utterance. 
Finally, PM s are said to not to affect the truth or falsity of an utterance. There¬ 
fore, although they may contribute somehow to the interpretation of an utter¬ 
ance, they are not an essential part of its meaning. 

Carter and M cCarthy (2006) provide a definition that takes these charac¬ 
teristics into consideration yet also places weight on the pragmatic contribution 
of these expressions. They discuss the connectivity and optionality of PM s, but 
they also highlight the importance of interpersonal meaning and the speaker's 
intentions in the definition. As part of their grammar of English based on a cor¬ 
pus of oral production data, Carter and M cCarthy (2006, p. 208) define PM s as 
"a class of items which operate outside of the structural limits of the clause and 
which encode speakers' intentions and interpersonal meanings." The authors 
subdivide the class of PMs into discourse markers (including such expressions 
as so, well, and anyway), stance markers ( actually, I think, of course), hedges 
(kind of, maybe), and interjections (gosh, wow). PM s are classified as grammat¬ 
ical options with social, contextual, and affective functions in spoken English. 

From this definition, the importance of PM s for interpersonal communi¬ 
cation is evident. These expressions allow speakers to communicate their inten¬ 
tions, indicate their attitudes towards information that is presented or received, 
and link their ideas for greater clarity. In other words, as Carter and M cCarthy 
(2006) point out, PM s are a broad class of items that can provide structure and 
organization to utterances while indicating attitude, assertiveness, or reactions 
to discourse (p . 105). 

In the next section, previous research relating to PM s in second language 
learning is discussed. The reader should keep in mind that the issues of the def¬ 
inition and identification make the comparison of different studies of PM s ten¬ 
tative. The literature review below outlines the findings of studies into the rate 
and range of use of PMs in second language research; however, the studies 
mentioned do not define or identify the markers in exactly the same way. 
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2 . 2 . Pragmatic markers in second language studies 

In most second language research on the rate of PM use, studies indicate that 
learners do not use PM s in their speech nearly as much as native speakers. Fung 
and Carter (2007) compared a corpus of elicited classroom data from interme¬ 
diate-advanced learners of English in Hong Kong with the British English CAN- 
CODE corpus. They found that non-native speakers largely underuse PMs in 
comparison to native speakers. Over half of the PM s they examined were used 
less frequently by non-native speakers. However, they acknowledge that their 
results must be considered preliminary due to the fact that the conditions under 
which the data were obtained were significantly different for native and non¬ 
native speakers. Romero-Trillo (2002) found similar results in his study of the 
use of a set of PM s by native and non-native speakers of English. Analyzing cor¬ 
pora of spoken data from Spanish native and non-native learners of English, he 
determined that native speaker adults used markers such as you know, I mean, 
or you see at a significantly higher rate than non-native speaker adults. 4 These 
findings are also supported by M uller's (2005) investigation of the use of so, like, 
well, and you know by native speakers and German learners of English. Analyz¬ 
ing data from a paired silent film retelling, she found overall significant un¬ 
deruse of these markers by learners (except for well). 

Although it is clear from previous research that the total frequency of PM s 
by non-native speakers of English is lower than that of native speakers, much re¬ 
cent research focuses only on a small set of expressions. Fuller (2003), Hellerman 
and Vergun (2007), M uller (2005), and Romero-Trillo (2002) looked at three, four, 
five, and six individual PMs, respectively. Such detailed analyses provide insight 
into the use of specific PM s in particular contexts and allow us to explore how 
their use by ESL speakers differs from that of native speakers. However, in order 
to gain an overall picture of the frequency, distribution, and variety of PMs in 
learner speech, it is necessary to widen the search and investigate a greater num¬ 
ber of expressions. Thus, this study attempts to fill a gap in previous research by 
investigating a larger variety of expressions from a comprehensive list of PM s. 5 

M ost studies of PM s focused on a single proficiency level or fail to address 
the role of proficiency at all. Fung and Carter (2007) focus solely on intermediate- 


4 In the case of look and listen, Romero-Trillo found no significant difference in the fre¬ 
quency of use by native versus non-native speakers. 

5 Note that Fung and Carter (2003) and Flasselgreen (2004) looked at a larger number of PM s 
than most studies. Fung and Carter identified 23 pragmatic markers among the 100 most com¬ 
mon English words and Flasselgreen looked at a total of 19 small words (her term for pragmatic 
markers). The current analysis expands on these larger-scale studies by focusing on even more 
expressions, indudingthe 121found in Carter and M cCarthy's (2006) corpus-based grammar. 
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advanced learners and also provide no comparative data from native speakers 
and nonnative speakers performing the same task. M uller (2005) examined four 
specific markers in detail but did not consider overall rates of discourse marker 
use, nor did she examine different proficiency levels. Romero-Trillo (2002) ana¬ 
lyzed data from both children (1st graders) and adult (university students) na¬ 
tive and non-native speakers but did not split up the non-native speakers ac¬ 
cording to proficiency. 

One of the few studies that addressed the issue of proficiency level in re¬ 
lation to a large set of PM s is Hasselgreen (2004). She focused on two groups of 
Norwegian learners of English (Norwegian A, described as "more fluent," and 
Norwegian B, or "less fluent") and determined that the frequency with which 
native speakers used smallwords was significantly higher than that of both non¬ 
native speaker groups. These groups were selected entirely on the basis of 
global grades on a speaking test (p. 160). Native speakers used 445 smallwords 
per 10,000 words while Norwegian A and B groups used them at a rate of 279 
and 235 per 10,000 words, respectively. Hasselgreen noted that these differ¬ 
ences were highly significant, with p < .0001 for native speakers versus Norwe¬ 
gian A and p <.05 for Norwegian A versus B (p. 171). 

Hellermann and Vergun (2007) also examined the interaction of profi¬ 
ciency level with PM use, although they limited their analysis to a small set of 
expressions. They investigated video recordings of classroom interaction and 
interviews of 17 adult learners of English with no previous formal English lan¬ 
guage instruction. Focusing on the PM s well, you know, and like, they analyzed 
the interaction of PM use and proficiency level. Proficiency was classified ac¬ 
cording to the four levels of English language courses available to the learners: 
from absolute beginner (Level A) to advanced (Level D). Hellermann and 
Vergun's (2007) findings indicated that the use of PM s for well, you know, and 
like went up with each proficiency level, although they found very few exam¬ 
ples, if any, at Level A and Level B (p. 167). 

Data on the interaction of proficiency level with the rate of PM use are 
potentially revealing if we consider the possibility that there might be a devel¬ 
opmental path in the acquisition of PM s. Both Scarcella (1983) and Hays (1992) 
speculated that learners acquire and use only certain PMs at lower levels of 
proficiency before acquiring others to use in more varied contexts at higher pro¬ 
ficiencies. This suggestion is supported by Hasselgreen's (2004) study: She pro¬ 
posed three hypothetical stages of development in the use of PM s that are ex¬ 
emplified by each group of speakers in her study (lower proficiency learners, 
higher proficiency learners, and native speakers). Certain smallwords are un¬ 
derused by the lowest-level speakers while other expressions are only part of 
the repertoire of higher-proficiency speakers or native speakers (p. 222). 
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Related to the role of proficiency and the possibility of a developmental 
path of acquisition in the use of PM s is the issue of the extent to which learners 
use a variety of PM s. W hen we focus on just a small set, the diversity is not clear. 
Richness could be related to proficiency level and development. As discussed 
above, previous research has found that learners generally use PM s at a lower 
rate than native speakers. However, as we will see below, PMs are also often 
overused by non-native speakers. 

Muller (2005) found that the markers so, like, and you know were un¬ 
derused by learners in comparison with native speakers. However, the marker 
well was not underused by non-native speakers. A full discussion of Muller's 
results is beyond the scope of this study since she also considered the effects of 
gender, age, relation between partners, role in the discourse, and time abroad 
on PM use. Relevant to this study is the fact that M uller's analysis found differ¬ 
ences in the rates of use of particularfunctions of PM s. For example, after iden¬ 
tifying 10 different functions for you know (five at the textual level, and five at 
the interactional level), M uller found that native speakers of American English 
used you know significantly more often than LI German speakers on all of the 
textual functions and three out of the five of the interactional functions. 

Hasselgreen (2004) also found a complex pattern of learners using differ¬ 
ent markers than native speakers to perform various communicative functions. 
In the category of hedges, for example, which "indicate the degree of vagueness 
or commitment" (p. 204), learners underused just, sort of/ kind of, like, and a 
bit. The "general extender" (Overstreet & Yule, 1997) type of hedge, exempli¬ 
fied by and stuff/and everything, was virtually absent from both Norwegian LI 
groups. Within the same category of hedges, the learners used / think and or 
something at higher rates than native speakers. What is more, the higher-pro¬ 
ficiency Norwegian group used more hedges than the lower-proficiency group, 
but not a greater variety. Romero-Trillo (1997) found that the markers / mean, 
well, you know, you see were underused but that listen was overused. He attrib¬ 
utes this to transfer of Spanish oye 'listen,' which is used frequently in Spanish. 

The current study aims to address several gaps in previous research by 
examining both the general rate of use as well as the range of use of PMs using 
data from learners at varying proficiency levels in comparison with native speak¬ 
ers. Previous studies have generally done one or more of these things, but not 
all at the same time. Hasselgreen (2004) is the only previous study that consid¬ 
ered the proficiency level of the learners and had comparable native speaker 
data on a larger set of PMs. However, the current study differs from Has- 
selgreen's in three ways that will contribute to the understanding of the devel¬ 
opment of learner PM use. The current study has a higher number of finely- 
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grained proficiency levels (5 vs. Hasselgreen's 2), uses monologic data rather 
than dialogic, and investigates a much larger set of PM s (121 vs. 19). 

It is important to consider both proficiency level and native speaker com¬ 
parison data when trying to build an understanding of aspects of learner 
speech. Foster and Tavakoli (2009) called for the inclusion of native speaker 
comparison data in more research studies. They argued that the inclusion of 
data from native speakers performing the same tasks under the same condi¬ 
tions is important in order to distinguish more clearly task and processing ef¬ 
fects from other effects. Likewise, data from learners at multiple proficiency 
levels allow for the investigation of how learner interlanguage develops. 

3. Research Questions 

This study was designed to investigate the acquisition of PM s by learners at vari¬ 
ous proficiency levels both in terms of the rate of their use as well as the type of 
expressions used. It therefore attempted to answer the following three research 
questions concerning the rate, frequency, and variety of PM s used by learners: 

1. W hat is the rate of PM use by ESL speakers at different proficiency levels 
and how does this compare to that of native speakers performing the 
same speech tasks? 

2. Do ESLspeakersand native speakers use the same PM s most frequently, 
even if they are generally alike or different in their rate of PM use? 

3. Do ESL speakers use a lesser variety of PM s than native speakers? 

After the methodology section below, the results of the quantitative analysis 
will be presented in the order corresponding to the order of these three re¬ 
search questions. 

4. Methodology 

4.1. The oral proficiency test 

The data for this study came from a computer-mediated, semi-direct test of oral 
proficiency (hereafter OPT) that is administered to prospective international 
teaching assistants. 6 The exam consists of ten items that require examinees to 
record their responses by speaking into a computer microphone. After receiving 


6 For more information on the methodology, participants, data coding, and testing procedure, 
see Neary-Sundquist (2013), where these issuesare discussed in the context of a similar study. 
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a prompt for each item, examinees have 3 minutes to take notes and plan their 
response. 7 Each response is limited to 2 minutes, after which time the recording 
is automatically stopped. 8 

For this study, four of the ten tasks were selected for transcription and cod¬ 
ing, including the "news," "personal," "passing information," and "telephone" 
tasks. 9 In the news task, an opinion about a news item must be given. In the per¬ 
sonal task, the examinees give a response to an open-ended audio question about 
their personal experience, such as how they learned English or who their favorite 
teacher is. In the passing information task, the examinees relate some infor¬ 
mation that they have read to someone who has no knowledge of it, such as de¬ 
scribing a job notice to someone they think might like to apply for the job. In the 
telephone task, examinees listen to a recorded message from one of their office 
mates and leave a short message on his/her home answering machine. 

The particular questions used in the different tasks in the OPT were not 
identical; rather, they came from several different forms of the test that are 
given regularly. The internal consistency coefficients on the different versions 
of the test are very high, ranging from .96 to .98, and the single factor solutions 
were found in a study of the factor structure (Ginther, Dimova, & Yang 2010). 

The exams are rated by two trained raters both of whom are also instruc¬ 
tors in the university's English proficiency program. When the two raters disa¬ 
gree, the exam is sent to a third rater to break the tie. The ten tasks in the exam 
are given a single holistic score that ranges from 2 to 6. A set of descriptors for 
each level can be found in Appendix A. 


7 The exam item prompts include written, video, and audio input; the exact text of the 
prompts is secured and consequently not available to the public. 

8 The monologic nature of the data used in this study could be considered a limitation in the 
analysis of PM s, due to their importance in interaction between the speaker and interlocutor. 
However, the testing circumstances under which the data were collected also offered a num¬ 
ber of advantages. First of all, there was no variability introduced by the presence of an inter¬ 
locutor. That is, the examinees were not more or less likely to employ a PM based on their 
judgment of the reactions of a conversation partner. Secondly, the testing environment al¬ 
lowed for the collection of larger amountsof data than have often been used in previous stud¬ 
ies of PM s. Thirdly, since the data examined in this study came from a language test, they had 
already been subdivided into proficiency levels whose reliability had been independently es¬ 
tablished. This makes it possible to investigate the use of PM s by varying proficiency levels, an 
aspect of learner pragmatics that has not been previously examined in the literature. 

9 These four tasks were chosen out of the ten available in order to provide tasks with a range 
of task types and levels of structure. The passing information and telephone tasks were 
considered more structured since they entail repeating specific information. The news and 
personal tasks were considered less structured since they are open-ended and the speaker 
has more leeway in structuring their response. 
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4.2. Participants 

Data consisting of 184 speech samples were analyzed. The ESL speaker exami¬ 
nees were all graduate students and prospective teaching assistantsat an Amer¬ 
ican university. The ESL speakers were grouped into four different proficiency 
levels, numbered 3 to 6. There were 10 participants at Level 3,10 at Level 4,10 
at Level 5,7 at Level 6, and 10 native speakers. All of the ESL speaker examinees 
came from either a Chinese or Korean language background. 10 The native 
speaker group consisted of speakers of American English who were also gradu¬ 
ate university students. A table showing the language background and scores of 
the participants can be found in Appendix B. 

Level 6 has only seven examinees because that number was the total 
available in the testing records. An additional limitation was that there were 
only six responses to the personal item for the native speaker group; the earlier 
version of the exam lacked this question. 

The data for each level was evenly split between examinees with a Chi¬ 
nese and Korean LI background; that is, there were 5 examinees with LI Chi¬ 
nese and 5 examinees with LI Korean in each group of 10. The group of 7 ex¬ 
aminees at Level 6 was made up of 3 native Chinese speakers and 4 native Ko¬ 
rean speakers. 

4.3. Data preparation and coding 

The oral exam responses were transcribed by the author using Carter and 
M cCarthy's (2006) list of PM s. Their definition is appropriate for this study since it 
was generated from a corpus-based investigation of actual language use. The cor¬ 
pus-based nature of the definition makes it particularly appropriate for this study 
since it likewise investigates a corpus of native and non-native discourse in context. 
Furthermore, this definition of PMs is particularly useful for methodological rea¬ 
sons: Carter and McCarthy's classification is accompanied by a list of 121 expres¬ 
sions. Unlike other studies that more narrowly define PM s and focus exclusively on 
a small group of PM s, Carter and M cCarthy's definition is more comprehensive and 
their list of appropriate expressions more extensive. Their list includes sub-categories 
of PMs, including "discourse markers" like you know, well, or so, "stance markers" 


10 These two language backgrounds were chosen based on several criteria. The first was that 
enough speakers of these languages take the exam in order to provide a range of participants 
at every proficiency level. The second consideration was that these groups learned English as 
a foreign language but their schooling does not take place in English, which might be the case 
for Indian learners of English. Two language groups, rather than one, were chosen in order to 
counterbalance the results against the possibility of transfer effects from the LI. 
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such as actually, of course, and hopefully, "hedges,"for example, / think, just, and kind 
of, as well as "interjections" like gosh, wow, or ouch. 11 

Two coders, the author and a colleague, analyzed the transcripts word- 
by-word for PM s. This methodology allowed for the identification of PM versus 
non-PM use of particular words. For example, so has multiple functions in Eng¬ 
lish, some of which are not PMs (as an adverb or conjunction indicating pur¬ 
pose), and some which are PM s (with functions such as introducing a summary, 
shifting the discourse, or even holding the floor; see Buysse, 2009 and Buysse, 
2012 for a thorough discussion of the various uses of so). Similarly, also can 
function as an adverb or as a PM that can signal a number of different discourse- 
related messages, including an additive function that is used when the speaker 
wants to legitimize the relevance of their utterance (see Waring, 2003 for fur¬ 
ther discussion on the functions of also in discourse). As an illustration of the 
multiplicity of uses, the passages below contain examples of so and also that 
were not counted as PM s, and further examples that were counted as PM s: 

... she's very worried so she's wondering whether ah she could setup a special time 
ah with you to ah get some help on homework ah before the next exam. (Korean 
Level 6 speaker) 

I have to take lessons, in English, and... many lessons in physics are also in English 
as well. (Chinese Level 3 speaker) 

... So, I’d say that it's, it's a good idea to involve university students, and give them 
interaction with schoolchildren. (Chinese Level 6 speaker) 

She asked to arrange a special time for her. If you can, she really appreciate you. Also, she 
worried about her homework because she did not do it well. (Korean Level 4 speaker) 

The first examples of so and also, underlined above, were not counted as PM s. 
The second set of examples of so (so I’d say) and also were counted as PM s. This 
use of so occurred at the end of the speaking passage, after several reasons for 
the speaker's opinion were enumerated. It marked a summative statement. Pros¬ 
ody also sometimes gave clues to the identification of this type of so (see below). 

A further advantage of the hand-coding methodology was that it was pos¬ 
sible to allow for some variation in the list of expressions. For example, Carter 
and M cCarthy (2006) list to sum up and in the end as PM s, but they do not in¬ 
clude in sum or all in all. Since the coders were reading through all of the data 
rather than electronically searching for expressions, it was possible to identify 


11 For a comprehensive list of all PM s and typical examples, see Carter and M cCarthy (2006). 
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such variations in wording. Pragmatic marker appears to be a highly idiosyn¬ 
cratic phenomenon in some cases, and this research methodology takes such 
individual variation into account. 

Interrater reliability for the coding of PM s was .94. In the case of a discrep¬ 
ancy in the identification of the PM s, the two coders discussed the exa mple i n ques¬ 
tion and came to an agreement. Coding posed several challenges at times; as Aij- 
mer (2013) pointed out, the meaning of PMs is inherently "flexible and fuzzy" (p. 
15). However, identifying PM s was aided by the use of a well-defined list (Carter & 
McCarthy, 2006). In cases where the raters still had doubts about whether a partic¬ 
ular expression was being used as a PM or not, listening to the actual recordings, 
rather than relying on the transcripts, was often helpful. Fung and Carter (2007, p. 
413) noted several prosodic "clues" that distinguish PMs: "pauses, phonological re¬ 
ductions, and separate tone units." Paying attention to such prosodic clues some¬ 
times helped the coders to identify PM uses of some words and expressions. As 
Aijmer (2013) noted, however, the relationship between the formal properties of 
PMs, such as prosody, and their function and meaning is complex; prosodic fea¬ 
tures cannot be relied upon exclusively to identify PM s. 

After the PM s were identified and counted, the total number of words used 
by each examinee was calculated. Non-lexical filled pauses such as um, ah, or uh 
were removed from the transcripts and not included in the total word count. The 
total number of words and the number of PM s used by each subject on each task 
were then entered into spreadsheets. The number of PM s was divided by the to¬ 
tal number of words to calculate the percentage of PM s that was used by each 
subject. The resulting percentages were then averaged to give the average per¬ 
centage of PM use for all of the responses at each level and for each task. 

4.4. Statistical analysis 

Descriptive statistics, including means, standard deviations, and 95% confi¬ 
dence intervals were calculated for the rates of PM use using Statistical Analysis 
Software, version 9.1.3. A Spearman rank order correlation was used to calcu¬ 
late the relationship between PM use and proficiency group since the OPT scale 
is ordinal. A Kruskal-Wallis test with Mann-Whitney post-hoc tests was per¬ 
formed to test the results for statistical significance. 

5. Results 

The quantitative results are presented in the following sub-sections according to 
the three main issues generated by the research questions. The data are followed 
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by a brief description of the results. The relationship between the results and the 
research questions will be addressed in the discussion section of the paper. 

5.1. Rate of pragmatic marker use 

The rate of PM usage was calculated by dividing the total number of words used 
by a speaker by the total number of PM s they used. Table 1 presents the de¬ 
scriptive statistics and 95% confidence intervals (CIs) for the rates of PM use for 
all five proficiency groups, along with the means and standard deviations of the 
number of words and PM s, and the mean rate of PM s for each level. 


Table 1 Descriptive statistics for the rate of PM use 


Level 

N 

Mean no, of 
words (SD) 

Mean no. of 
PMs(SD) 

Mean PM rate (SD) 

Min 

Max 

Lower 
95% Cl 

Upper 
95% Cl 

3 

10 

421 (79) 

9(5) 

0.021(0.01) 

0.003 

0.035 

0.01 

0.03 

4 

10 

525 (144) 

13(10) 

0.025(0.019) 

0.006 

0.066 

0.01 

0.04 

5 

10 

579(122) 

18(10) 

0.031(0.016) 

0.012 

0.064 

0,02 

0.04 

6 

7 

743 (116) 

40(21) 

0.052 (0.02) 

0.037 

0.094 

0,04 

0.07 

NS 

10 

584 (215) 

31(13) 

0.052(0.014) 

0.025 

0.067 

0.04 

0.06 


Note. NS = native speakers 


The results show that PM use rose with proficiency level; however, the pattern was 
not entirely straightforward. Levels 3 and 4 were similar, at 2,1% and 2.5%. The 
sharpest gain in PM use occurred at levels 5 and 6, with a jump of 1% and then a 
further jump of over 2%. But PM usage did not continue to rise for the native speak¬ 
ers; instead, their usage was identical to that of the Level 6 group. The pattern 
showed that the two highest and the two lowest proficiency groups were very alike 
in their PM use but that the middle level showed a steeper rise in PM use. 

Correlation tests and tests of statistical significance indicated that there 
was a positive correlation between PM use and proficiency level and that there 
were significant differences between levels. The Spearman correlation for the 
rate of PM use and proficiency level was .66. The results of a Kruskal-Wallis test 
were significant (H =22, 4 df, p =<0001), Mann-Whitney post-hoc pairwise 
comparisons showed that Level 3 was significantly different from Levels 5, 6, 
and native speakers, and that Levels 4 and 5 were each significantly different 
from Levels 6 and native speakers. 

The results of the analysis of PM usage rate can be summarized as follows: 
The use of PM s correlated positively with proficiency level. Learners generally 
used PM s at a lower rate than native speakers, but advanced learners' rate of 
use did reach native-speaker levels. The use of PMs increased gradually from 
Levels 3 to 5 and rose rather abruptly between Levels 5 and 6. 
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5.2. Frequency of particular pragmatic markers 

The second research question is: Which particular PMs were used most fre¬ 
quently both by learners and by native speakers? Table 2 shows the three most 
frequently used expressions for each level. Results indicate that the three most 
frequently used expressions were the same for the Level 3,4, and 5 proficiency 
groups. I think, so, and also were the top three PM s in these three groups. The 
Level 6 group is noticeably different in this regard, with just one of these ex¬ 
pressions in the top 3: so. The native speakers had two of the same expressions 
as the first three proficiency groups: / think and so. In terms of differences in 
the groups, Level 6 and the native speaker group stood out for their frequent 
use of just, while the Level 6 results were notable for the inclusion of you know. 

Table 2 The three most frequently used PM s for each proficiency level 


Level 3 

Level 4 

Level 5 

Level 6 

Native speakers 

/ think 

1 think 

1 think 

just 

/ think 

so 

also 

so 

so 

so 

also 

so 

also 

you know 

just 


Table 3 provides an overview of these five most common PMs in the corpus at 
each of the five proficiency levels. Both the raw figures for the number of PMs 
used and the percentage each PM represents are given. The percentages for each 
particular PM were obtained by dividing the number of occurrences of that 
marker by the total number of PMs. The data indicate that the higher proficiency 
levels, including Levels 5 and 6, along with the native speakers, had a more evenly 
distributed use of different PM s. Speakers at lower levels (Levels 3 and 4) used 
certain PMs, including / t/o/'n/cor so or also for Level 4, at a much higher rate than 
other more common phrases like just or you know. Thus, the higher levels of pro¬ 
ficiency showed a flatter distribution of the most frequently used PM s, while the 
two lower levels exhibited some spikes in the frequency of certain PMs. 


Table 3 The frequency of particular PM s by proficiency level 


PMs 

Level 3 

Level 4 

Level 5 

Level 6 

NS 

Total PMs(%of PMs) 

92 (100) 

134 (100) 

181 (100) 

281 (100) 

310 (100) 

1 think 

30 (32.6) 

31(23,1) 

32(17.7) 

25 (8.9) 

44(14.2) 

so 

16 (17.4) 

10(7.5) 

26(14.4) 

31(11) 

28(9) 

also 

12 (13) 

25(18,7) 

21(11.6) 

19 (6.8) 

15 (4.8) 

just 

3(3.3) 

5(3.7) 

7(3.9) 

31(11) 

23 (7.4) 

you know 

1(1.1) 

5(3.7) 

14(7,7) 

29(10.3) 

10 (3.2) 


Note. NS = native speakers 
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5.3. The variety of pragmatic marker use 


In the previous section, results indicated that learners in the lower proficiency 
groups were overusing certain expressions such as / think in relation to the higher- 
proficiency speakers. If this is the case, then this lack of variety in their expressions 
should be apparent in more than just one expression. Figure 1 shows the number 
of different PM s used by learners in each proficiency group. The number of dif¬ 
ferent expressions used at each level went up steadily from 25 PM s at Level 3 to 
69 PM s among the native speakers. This shows that as learners acquire PM s, they 
also make use of a greater variety of them. However, it is possible that, although 
they were using a greater variety of expressions, these new expressions did not 
represent a large proportion of their overall PM use. In order to examine if this 
was the case, it is necessary to look at how much of the learners' PM use is ac¬ 
counted for by the most frequently used expressions at each level. 


80 

70 

60 

50 

40 

30 

20 

10 

0 



Level 3 Level 4 Level 5 Level 6 Native 

speakers 


Figure 1 The number of unique PM s used by level 

Figure 2 compares the use of the three most frequent PM s for each pro¬ 
ficiency group with the remainder of the PM each group used. The data here 
show that the percentage of PM s accounted for by the most used expressions 
fell consistently with proficiency level. That is, PM use for the lower proficiency 
groups was composed of repeated use of a few expressions and a more limited 
use of a larger variety of expressions. The three most frequent expressions ac¬ 
counted for a majority of PM use for the Level 3 proficiency group and almost 
half for the Level 4 proficiency group. 


651 



Colleen Neary-Sundquist 


■ 3 most used PM s □ All other PM s 



Level 3 Level 4 Level 5 Level 6 Native 

speakers 


Figure 2 The percentage of PM s accounted for by the three most frequent PMs 

6. Discussion 

6.1. Frequency of pragmatic markers across proficiency levels 

According to the results presented above, the data indicate that PM use rises 
with proficiency level and correlates positively with it. At first glance, the fact 
that this is the case is not surprising. It is expected that PM use would rise with 
proficiency level, based on previous research on PMs among second language 
learners (Hasselgreen 2004; Hellerman & Vergun, 2007). The use of PM s shows 
that the speakers are able to provide additional information about their mes¬ 
sage, in addition to its basic meaning. This may indicate that they do not have 
to allocate as much attention to formulating their basic message and can in¬ 
stead give some attention to how they would like it to be interpreted. This is 
reflected in increased PM use at higher proficiency levels. Thus, in terms of the 
Research question 1 of this study, the data indicate a general rise in the fre¬ 
quency of PM s with increased proficiency. 

What is surprising, however, is the abrupt rise in PM use between Levels 5 
and 6. This seems to indicate that PM use makes sudden progress after a certain 
critical mass of general proficiency is reached. During the transition from Level 5 
to Level 6, the learners' use of PMs jumps dramatically to native-like levels. This 
seems to indicate that this area of pragmatic development occurs fairly late for 
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learners. This could possibly be due to the fact, often mentioned in definitions of 
PM s, that these expressions are part of the message but not of the essential prop¬ 
ositional meaning. Therefore, learnersare only able to add these types of nuances 
to what they are saying when they do not have to devote as much attention to 
formulating the content of what they are saying. Looked at from the perspective 
of PM s and fluency, this could also indicate that the use of PM s is actually sup¬ 
porting or even enabling the speakers to become more fluent. However, this sug¬ 
gestion must be considered with caution in light of the fact that this study does 
not directly examine the relationship between PM use and fluency measured with 
temporal variables. The relationship between PM s and fluency is no doubt com¬ 
plex and provides rich opportunities for further study. 

Although a sharp increase in the rate of PM use is seen between Levels 5 
and 6, the use of PM s was still notable at Levels 3 and 4. Even at the fairly low 
proficiency level (Level 3), the examinees used 2.1% PMs. This is remarkable 
when we consider that PM s are not regularly taught in the classroom. It is likely 
that this pedagogical oversight is due to the fact that PM s are considered sec¬ 
ondary, extraneous, and optional compared with the necessity of mastering cat¬ 
egories such as verbs, nouns, and prepositions. It may also arise from the fact 
that PMs are considered too difficult to teach. With few formal instructional 
materials, teachers are left to rely on their own intuition, which they soon find 
inadequate to explain the intricacies of PM use. 

6.2. The most frequent pragmatic markers at each proficiency level 

In terms of Research question 2, the results indicate that the first three profi¬ 
ciency groups were remarkably similar with respect to the PM s they used most 
frequently. At least at this level of analysis, the results are somewhat surprising 
in how similar they are across groups; we might have expected at least some 
variation in the most frequent expressions. However, the pattern that emerges 
instead is one of close similarity among the first three levels and then a drastic 
difference at Level 6, with only one of the same expressions to be found among 
the three most frequent PMs. Interestingly, this pattern mirrors the pattern 
seen above for the frequency of PMs in general. In both cases, the first three 
proficiency groups cluster together, followed by a jump in use for Level 6 and 
the native speaker group. These results therefore also support the intriguing 
pattern that some significant change in the acquisition of PM s occurs between 
the Level 5 and Level 6 proficiency groups. 

It is important, however, to note that the native speaker group has two 
expressions in common with the three lower proficiency groups; this argues 
against the significance of the fact that the Level 6 group differs markedly with 
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respect to the proficiency groups below it. Note that these expressions do not 
represent the most frequent expressions for all of the groups. Rather, they were 
included based on the fact that they appeared in the top three most frequent 
expressions for any one group. 

As we can see in Table 3 above, the three highest proficiency groups, Level 
5, Level 6, and native speakers, are quite similar in their rates of use of these 
five expressions. The Level 3 proficiency group seems to follow the same gen¬ 
eral trend as these three higher proficiency groups, although the rates of use 
are quite different for / think. The Level 4 proficiency group is the outlier here, 
with noticeably higher rates of use for / think and also. 

W hat the two lower proficiency levels have in common is that they both 
use / think to a much greater extent than the three higher proficiency groups. 
There are several possibilities that would account for greater use of / think. It is 
possible that they are using this expression as a filler, much in the same way a 
speaker might repeat umm. Other possibilities are that they could be marking 
greater uncertainty about what they are saying, or they may be overusing this 
PM to compensate for the fact that they do not have a command of a greater 
variety of expressions. 

Recall that Hasselgreen (2004, p. 222) also discussed the possibility of an 
order of acquisition for PM s based on which PM s the lower and higher proficiency 
groups in her study seemed to have mastered. Her results are not directly com¬ 
parable to those in this study due to the differences in the types of tasks from 
which her data were drawn. However, she too observed two main stages of de¬ 
velopment that resemble those found in this study: Learners at lower levels of 
proficiency tend to overuse a small number of expressions, while those at higher 
levels expand to a greater variety that resembles the pattern of use among native 
speakers. The possibility of such changes in PM use would have important impli¬ 
cations both for the study and teaching of PM s and requires further research. If it 
is true that there is an order of acquisition for PM s, this would not be the first 
pragmatic phenomenon that has shown evidence of such an order. Dittmar and 
Terborg (1991), for example, found an order of acquisition for expressions that 
encode modality. Similarly, Kasper and Rose (2002) suggested that the act of mak¬ 
ing requests proceeded through five stages for L2 learners. 

As pointed out in the methodology section, this study relies on monologic 
data, which has some advantages in terms of research design, allows for the 
analysis of a large amount of data for cross-sectional comparison, and makes it 
possible to consider PM use by learners at independently established profi¬ 
ciency levels. However, another effect of the monologic data may be that cer¬ 
tain markers, such as you know, are likely to be less frequent since there is no 
interlocutor present. However, it is interesting to note that many of the PMs 
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that signal the speaker's attitude or offer indications about how their utter¬ 
ances should be interpreted are still well represented in the results. For exam¬ 
ple, hedges such as / think were still quite frequent among learners at all levels. 
This suggeststhat even in a monologic environment, the speakers are still giving 
cues to an imagined interlocutor, at least in the circumstances under which the 
data were collected. Even learners at a low proficiency level feel the require¬ 
ment to offer this type of pragmatic information, which may indicate some in¬ 
teresting avenues for future research into learner pragmatics. 

6.3. Variety of pragmatic markers at each proficiency level 

In terms of Research question 3 and the variety of PM use, ESL speakers use 
fewer unique expressions than native speakers. In contrast to the rate of use of 
PMs in general, which rose sharply between Levels 5 and 6, the number of 
unique expressions used rose more gradually from 25 expressions at Level 2 to 
58 at Level 6. The largest gain in the number of expressions used occurred be¬ 
tween Levels 4 and 5, where 17 new expressions were gained. Although the 
Level 6 group used PM s at the same rate as native speakers, they were still well 
behind them in the variety of expressions. 

The data on the variety of PM s further suggests the interesting possibility 
that learners first attain command of a critical mass of expressions before they 
begin to use them at more target-like levels. This increase in the number of 
unique expressions used thus occurs earlier than the increase in the rate of PM 
use, which, we saw, occurred between Levels 5 and 6. The fact that an increase 
in the variety of expressions used precedes an increase in the rate of use is log¬ 
ical and might indicate that a critical mass of PM s must be mastered before ESL 
speakers might begin to use them at native-like levels. 

One interesting possibility concerning the relationship between the rate 
and the range of PM use is suggested by research on second language vocabu¬ 
lary acquisition. Various studies of vocabulary acquisition have found a con¬ 
sistent relationship between general proficiency level and measures of lexical 
diversity (Daller, Van Hout, & Treffers-Daller, 2003; M alvern & Richards, 2002; 
O'Loughlin, 1995; Zareva, Schwanenflugl, & Nikolova, 2005). Lexical diversity 
has proven to have a more consistent positive correlation with proficiency level 
than other measures of vocabulary acquisition. 

Lu (2012) found that a number of different lexical variation measures had 
significant correlations with the quality of oral English narratives produced by 
Chinese learners on proficiency test rankings. It was lexical variation, rather 
than lexical sophistication or density, that exhibited this correlation. Yu (2010) 


655 




Colleen Neary-Sundquist 


also found a statistically significant positive correlation between lexical diversity 
and the ratings given to writing and speaking samples of learners of English. 

The acquisition of PM s might proceed in a fashion that is similar to that 
of vocabulary, with learners showing an overreliance on a more limited number 
of expressions. 

7. Conclusions, limitations, and directions for future research 

This study provides new data on the general rate of PM use in the speech of ESL 
learners. The data indicate that the frequency of PMs increases gradually at 
first, increases more dramatically at the highest proficiency levels, and reaches 
a rate identical to that of native speakers. Learners at lower levels of proficiency 
tend to use only a small repertoire of PMs that includes / think, so, and also, 
while those at higher levels use more varied expressions at rates similar to those 
of native speakers. The richness of these expressions increases gradually across 
proficiency levels but never reaches native-like patterns of variation. 

This study was limited by several important factors that offer indications 
of areas for future research. The data used in this study come from a semi-direct 
test of oral proficiency. The fact that the data come from a language test and 
are monologic (no interlocutor was present) may have a noticeable impact on 
the types of language produced. There are few studies that directly compare 
the same tasks being performed under both monologic and dialogic conditions, 
and those that have examined this have shown mixed results. Gan (2012) found 
that ESL learners produced more T-units and clauses and longer T-units and ut¬ 
terances on monologic rather than dialogic tasks. M ichel, Kuiken, and Vedder 
(2007) found that the language produced by learners of Dutch on monologic 
tasks was more complex but less fluent and accurate than the language pro¬ 
duced on dialogic tasks. Therefore, the monologic condition under which the 
data for this study were gathered may have a significant impact on the results. 
Further research should look at PM use in more naturalistic data and with a 
speaking partner. This is especially important given the interactive nature of 
PMs. As discussed above, the monologic data proved to be advantageous in 
many ways in the research design of this study; however, dialogic data would 
provide additional information concerning aspects of PM use that are not ad¬ 
dressed in this study. 

As discussed above, another limitation involved in the study of PM s is the 
lack of agreement about their function, status, and identification. The identifi¬ 
cation issue is particularly troublesome since it affects the comparability of dif¬ 
ferent studies. This study has tried to minimize the problem by using a compre- 
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hensive, published list of PM s from a corpus-based grammar (Carter & M cCar- 
thy, 2006). However, the generalizability of the results here isstill limited by the 
fact that most studies use different definitions, lists, and even terminology for 
PM s. Building an understanding of how learners use PM s will require not only 
more research but particularly research that makes it clear which types of ex¬ 
pressions were counted as PMs. 

The findings of this study that highly proficient non-native speakers use 
PMs at a rate, but not with a range, similar to native speakers also suggests 
areas for future research. There may be potentially instructive comparisons to 
be made between the use of PMs and the development of vocabulary in gen¬ 
eral. Research into second language vocabulary acquisition, as discussed above, 
has shown that lexical diversity correlates positively with proficiency level. It 
would be instructive to consider the development of PM use in comparison with 
the development of lexical diversity and to examine whether they proceed in a 
similar fashion. In fact, the development of PM use should be considered in light 
of the growth of other linguistic subsystems in learners' interlanguage. 

In this vein, Bardovi-Harlig (1999) noted a lack of an acquisitional perspec¬ 
tive in second language pragmatics and has called for a research agenda that 
re-focuses attention on the interlanguage in interlanguage pragmatics. One ex¬ 
ample she gives is the investigation of expressions that include / think and how 
the acquisition of these expressions might or might not be related to the acqui¬ 
sition of the grammatical knowledge necessary to use them. This is also a signif¬ 
icant next step necessary for expanding our understanding of how PM use fits 
into the larger project of second language pragmatic development. 
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APPENDIX A 


Descriptors for the levels of the oral proficiency test 


Level 

Content 

Delivery 

6 

Wide range of vocabulary 

Complexity of sentence structure 

Interpretative/summary statements 

Some non-native usage 

M eaning clearly expressed 

Provision of a frame 

Economy of expression 

Smooth delivery 

Almost no pauses/hesitations/ choppiness 
Thought expressed in one utterance 

No problems with articulation 

Use of varied intonation and tone 

5 

Somewhat unconventional words 

Listener effort needed at times 

Simple sentence construction 

Well organized and coherent 

Meaning clear 

Clearly non-native like delivery 

Some pauses and choppiness, but compre¬ 
hension unobstructed 

Some sound substitutions 

Listener effort required at points 

4 

Dependence on the prompt 

Ineffective/abrupt transitions 

Omission of function words 

Systematic problems with bound morphology 
Topic shifts 

Lack of coherence 

Weak organization 

Repetition interferes with coherence 

Intended meaning unclear 

Lack of elaboration 

Ineffective repetition of words/ phrases 
Pauses/hesitations are more frequent 

Flat intonation 

M any identifiable articulation/ pronuncia¬ 
tion/stress problems 

Pace interferes with comprehension 

Close listener attention required 

3 

M isuse of particular words 

Problems with bound morphology 

Frequent attemptsto re-start/ 
re-phrase without clarification 

Unintended meaning 

M isunderstands prompt 

Deliberate/ ineffective delivery 

Frequent pauses/hesitations within 
phrasal boundaries 

Ineffective attempts of interpretative state¬ 
ments 

Limitation of vocabulary 
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APPENDIX B 


Participant data 


Number 


Native language 


Native country 


Oral proficiency test 
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APPENDIX C 

List of pragmatic markers from Carter and M cCarthy (2006) 


actually 

1 mean 

not to say 

that's to say 

admittedly 

1 must admit 

now 

there again 

also 

1 must say 

obviously 

third/ ly 

amazingly 

1 think 

of course 

to be frank 

and 

I'm afraid 

oh 

to be honest 

and then 

I'm sorry 

okay 

to put it another way 

anyway 

ideally 

on top of that 

to put it mildly/bluntly 

apparently 

ifyouaskme 

000 

to sum up 

arguably 

if you like 

oops 

to tell you the truth 

as 1 say 

in a manner of speaking 

or rather 

ugh 

as 1 was saying 

in fact 

ouch 

understandably 

as it were 

in general 

perhaps 

undoubtedly 

basically 

in other words 

predictably 

unfortunately 

by any chance 

in the end 

presumably 

well 

certainly 

in the X place 

probably 

what's more 

clearly 

indeed 

putting it mildly/bluntly 

where was i 

confidentially 

just (about) 

really 

wow 

'cos 

just think 

remember 

you know 

damn 

kind of 

right 

you see 

doubtless 

last of all 

rightly 


essentially 

lastly 

roughly 


finally 

like 

sadly 


fine 

like 

second/ly 


firstfof all/ly) 

listen 

seriously 


for a start 

literally 

so 


fortunately 

look 

so to speak 


frankly 

maybe 

sort of 


going backto 

mind you 

strictly speaking 


good 

naturally 

surely 


gosh 

next 

surprisingly 


great 

no doubt 

thankfully 


honestly 




hopefully 
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